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"Gradient and Full-Response Feedback in Computer-Assisted Instructions" did not 
substantiate the hypothesized advantages of gradient-response over full response 
feedback on the basis of amount learned (n=24). "Relative Effectiveness of Various 
Modes of Stimulus Presentation through Computer-Assisted Instruction" reported no 
significant differences for audio, typed, or display modes of presentation, but all 
three were significantly better than no instruction (n=90). "Expressed Student 
Attitudes under Several Conditions of Automated Programed Instruction" reported 
the use of "contingent prompting and feedback" and "knowledge of correct response" 
computer-assisted instruction and a programed text all containing the same program 
with 66 students in grades 9 and 10* A 40-item attitude scale revealed both methods 
resulted in significantly greater positive attitude than did the programed text. 
"Remedial and Review Branching in Computer- Assisted Instruction" contrasted 
branching and nonbranching programs. Results from 42 students indicated no 
differences in learning but the branching program required more time. Two projects 
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CAI Student Station" and "CAI Time Accounting and Usage Analysis" are reported. VT 
005 978 is a similar report. (EM) 



o 

ERIC 




COMPUTER ASSISTED INSTRUCTION LABORATORY 



COLLEGE OF EDUCATION • CHAMBERS BUILDING 





( 



The Pennsylvania State Univerfr* H : > . i- 

^ f 

Computer Assisted Instruction Laboratory 
- University Park, Pennsylvania 



Semi-Annual Progress Report 

EXPERIMENTATION WITH COMPUTER-ASSISTED INSTRUCTION 
IN TECHNICAL EDUCATION * 

Project No. 5-85-074 



principal Investigators 
Harold E^Mitzel ^"George L. Brandon 



Associate Investigators 
Keith A. Hall, Laboratory Director 
Betta H. Kriner, Admini strati :e Assistant 

Terry A. Bahn Joseph' L. French 

Bobby R. Brown David A. Gilman 

Donald W. Johnson 



Contributing Authors and Researchers 

Marilyn Adams Clara Gargula 

Karl G. Borman John Tardibuono 



June 30, 1967 
Report No. R-6 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POLICY. 



FOREWORD 



This report marks the end of the second year of a four-year research 
and development effort Into one of the relatively unexplored new technolo- 
gies of education. A good part of our first year was spent in learning 
how to manipulate, program, and maintain unfamiliar equipment. During the 
second year just ended, we have Increased the number of curriculum segments 
available for technical education learners at the CAI terminal, investi- 
gated some of the research questions of greatest interest, and expanded 
our CAI teleprocessing system to include two terminals at our sister insti- 
tution, the Williamsport Area Community College, as well as two terminals 

on the Altoona Campus of the University. 

Unfortunately, due to budget cuts for the current fiscal year, we 
have reluctantly had to withdraw the terminals from Williamsport and Altoona. 
Both institutions provided us with unstinting cooperation in the selection 
of technical education students who served as subjects. The faculties of 
both schools gave us many hours of valuable consultation on curriculum and 
display problems. The executive officers of our two cooperating institu- 
tions, President Kenneth Carl at the Williamsport Area Community College 
and Mr. Robert Eiche, Director of The Pennsylvania State University at 
Altoona, have been patient, supportive, and understanding of the special 
requirements for educational research. For their help we are most grateful. 
Mention should be made of the efforts of Mr. George Wolfe at Williamsport 
and Mr. Robert Smith at Altoona who bore the brunt of our unreasonable 
demands and good-humoredly tolerated, our intrusions into the calm of their 
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academic lives. 



During the next twelve months we will be primarily In another "hard- 
ware" phase as we accommodate to the new IBM 1500 Instructional System. 
Delivery Is anticipated In November, but the spring of 1968 should find 
us producing research on some new questions after translating a significant 
portion of the curriculum Into Coursewrlter II codes and adapting displays 
to the cathode ray tube. 



University Park, Pennsylvania 
June 30, 1967 



Harold E. Mltzel 
George L. Brandon 
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GRADIENT- AND FULL-RESPONSE FEEDBACK 
tn COMPUTER-ASSISTED INSTRUCTION 



Keith A. Hall, Marilyn Adams, and John Tardlbuono 



It is not appropriate at this point of development to maintain that 
computer-assisted Instruction (CAI) can or does provide more instruction that 
is more efficient or effective than conventional methods. However, it Is 
appropriate to scrutinize the particular characteristics of a given system 
to determine whether or not there can be or is improvement In learning. 

Many of these characteristics or variables cannot be judged in terms of 
previous experimentation in the field of educational psychology because they 
exist only in CAI. Further, studies conducted in laboratory situations can- 
not be readily transferred to an educational environment. This study 
attempted to remove one learning experiment from the artificial world of 
animal laboratories and nonsense syllables and to place it in an educational 

context. . 

Lumsdalne (1962) suggested two or three Important gradients in programed 

instruction, one of which Is an intraframe gradient Involving the possibility 
of progressively Increasing prompt strength within a frame from zero to the 
point at which the student is able to respond. Exploratory experimentation 
by Angel 1 and Lumsdalne (1961), Israel (I960), and Guthrie and Lumsdalne 
(1961) suggested that a satisfactory way of adjusting these gradients might 
require Instrumentation with considerable versatility; this Is one of the 
areas in which experimentation with computer-assisted Instruction might be 
of considerable Interest. Angell and Lumsdaine (1960) also found that 
complete prompting was not entirely desirable. Their results showed that, 
contrary to the interpretation of previous data, learning was significantly 
more efficient under the condition of Incomplete or aperiodic prompting 
(prompting on three out of four trials) than under conditions of complete 

prompting. 

The Coursewriter language used at Penn State University in the develop- 
ment of CAI course materials has two functions (among others) for providing 
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feedback to the student regarding the correctness of his response. Each 
of the functions gives the author flexibility in selecting the type of feed- 
back to provide the student. The two functions under investigation were 
keyword (kw) and partial -answer zero (paO). The kw function can be used to 
match a student's response against a stored correct response which may be a 
complete word or several complete words. The paO function causes the system 
to match a student's response against the stored correct response, one or 
more characters at a time, depending upon the author's choice. In each case 
feedback can be given to the student based upon what was and/or was not 
matched. 

The simplest form of feedback with kw is the complete response which 
the student should have made. The simplest form of feedback with paO consists 
of the characters which matched correctly, dashes for lowercase characters 
which did not match, and underscores for uppercase characters which did not 
match. A typical student-system interaction with paO and kw is shown in 
Figure 1. It should be noted that all student responses here are italicized. 
The CAI system typed everything else which appears in the example. In the 
first instance, the computer presented the stimulus item "North Dakota" to 
which the student should respond with the state capital. In the partial- 
answer processing example, the student first responded with Raleigh. The 
system matched this response character by character against the stored 
correct response and typed an underscore for the missed capital letter, 
dashes for the other missed letters, and the letters applicable to the 
correct programed response rearranged in appropriate order. A frequency 
count of the characters in all of the correct responses in the program 
indicated that typing clues and hints or more clues and hints would provide 
most of the essential characters for the student in his answer. The stu- 
dents were told to type one of these phrases if they could not think of the 
correct answer. The student then was presented with a few of the characters 
in the correct response. Notice that the character r which he had matched 
previously was not included in clues and hints and therefore did not appear 
in the feedback from the computer. After the student typed more clues and 
hints , he was given sufficient feedback to enable him to type the correct 
answer. 
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Partial -Answer Processing 

North Dakota 
Raleigh* 

J--ar~ 

clues and hints 

_is-a-c- 

more elves and hints 

jismarc- 

Bismarek 

Wisconsin 
Racine 
ja-1 — i n 

clues and hints 

_ad1s-n 

Madison 

Oklahoma 
Oklahoma City 



Acquired: 8 Total Trials: 

Time: 49608.08 seconds 



Keyword 

North Dakota 

Pierre 

Bismarck 

Bismark 

Bismarck 

Bismarck 



Wisconsin 

(BOB) (No typewritten response) 
Madison 

Madison 



Oklahoma 
Oklahoma City 



50 Total Responses: 94 



Total response time: 794.66 



♦Italics Indicate the responses typed by the student; all other material was 
typed by the CAI system to the student. 



Fig. 1. Sample of student-systems Interaction on partial -answer 
processing and keyword experimental treatments. 
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Looking at the same sequence in the kw treatment, the student typed 
Pierre and the system responded with the word "Bismarck." The student was 
then required to type the correct response before he could proceed to t*a 
next stimulus items. However, on the student's first attempt he misspelled 
the word; therefore he got the same feedback and had to retype the response. 

The desirability of less than complete prompting found by Angel 1 and 
Lumsdaine (1960) and the suggestion of providing a prompt based on the 
ability of the student to respond correctly (Lumsdaine* 1962) are variables 
potentially of great importance for improving instruction. However, the 
specific technique of adjusting the variables have not been adequately 
Investigated. This is due in part to the lack of technology to properly 
manipulate the instructional sequence. The characteristics of the paO and 
the kw function provide the necessary control and flexibility. 

Method 

Stimul us Materials 

Programs were written for investigating the effectiveness of these two 
kinds of feedback — gradient and full -response feedback. The flowchart 
for the courses used in the experimental treatments is presented in Appen- 
dix A. The programing and flowchart for a specific question appears in 
appendices B, C, and D. A paired-associate learning task was employed 
using fifty pairs which the student learned. The fifty state names of the 
United States were presented as stimulus items, and the student learned to 
respond with the names of the capitals.^ The items were presented individ- 
ually, in random order, to the student at the typewriter terminal. If the 
student responded correctly (acquired) on his first attempt to that stimulus, 
it would be dropped from the program. The program recycled until each stu- 
dent had responded correctly on his first attempt to each of the items during 



Hhe authors deliberately chose a learning task about which the 
subjects would have some partial information prior to the experiment. 
This choice of material parallels a typical classroom learning situation 
because learners are rarely ever completely ignorant of a subject taught 
to them in classrooms. 
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one cycle. The number acquired on the first cycle through the program was 
used as a pretest score. After the student acquired each of the fifty pairs, 
the entire list was presented as a posttest. A retention test was adminis- 
tered to each student two weeks after the Initial treatment. The program 
for each of the experimental treatments contained the following features. 

1 A list of warm-up items consisting of five foreign countries 
presented as stimuli and their capitals as response items. 
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A typing test which recorded the student's ti ^ e 7 ® n ^J;!J^ cy 
in typing an alphabetic sentence consisting of 74 computer 
characters. (A sample of student-system interaction cn the 
typing test is presented in Appendix E.) 



3. 



A progress report to the student after each cycle (see Figure 1) 
consisting of 



a. 

b. 

c. 

d. 

e. 



total number of responses 
total number of stimuli presented 
total number of items acquired 
total response latency 
current clock reading 



4. An automatic 5-minute break approximately halfway through 
the task. 



5. An automatic connection to a system-administered student 
opinion survey (SOS) regarding CAI. 



Subjects 

On the basis of variability data collected during a pilot study, it 
was estimated that a minimum of ten subjects In each experimental group 
would be needed for statistical purposes (Hayes, 1965). A total of 24 
students from the Williamsport Area Community College and the Altoona 
Campus of the University were Included in the study and were randomly 
assigned to the experimental treatments. Remote terminals are maintained 
at each of these locations for the purpose of conducting experimental 
studies which Involve vocational and technical education students. 
Proctors who administered and supervised the students at these locations 
were given specific Instructions to read to the students participating in 

the study. 
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Procedures 



CAI proctors from the two campuses solicited volunteers to participate 
in the experimental study. The students were told that the study would 
require a maximum of four hours and that they would be paid a flat rate 
of $5.50 regardless of how much actual time it took them to complete the 
study. The students were randomly assigned to one of the experimental 
treatments. The proctors assisted the students with the initial registra- 
tion procedure. The students’ typeouts were returned to the authors for 
analysis. 



Variables 

The following summarizes the variables and their parameters which were 
of concern in this study: 

1. Pretest, posttest, and retention tests: all contained the same 
items — the names of the 50 states of the United States with a 
possible high score of 50 and a possible low score of 0. 

2» Total number of stimuli presented: fifty stimuli (state names) 
were presented to each subject on the firs* cycle through the 
material. Those to which he responded c. ctly on his first 
attempt were not presented again. Those to which he responded 
incorrectly were retained and presented again during the next 
cycle of the program. The minimum number presented was 50; the 
maximum was unlimited, determined by each subject's performance. 

3. Total number of responses during instruction: each time a stimulus 
was presented a response was required from the subject. Feedback 
was presented after each incorrect response and additional responses 
were required until the correct response was made. The minimum 
number of responses was 50; the maximum was unlimited but based 

on each subject's performance. 

4. Total instructional time: includes the elapsed time from when 
the first item was presented until the final correct response was 
made. 

5. Student opinion survey: a 20-item multiple choice questionnaire. 
Each response was weighted from five to one to produce a maximum 
score of 100 reflecting a strong favorable attitude toward CAI 

or a minimum score of 20 reflecting a strong negative attitude 
toward CAI. 
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6. Response latency during instruction, posttest, and retention test: 
After each stimulus item was presented a green PROCEED light turned 
on. The PROCEED light remained on until the subject pressed the 
EOB key to record his response. The time interval that the PROCEED 
light was on was recorded as the response latency for that item. 

The sum of these intervals was the total response latency accumu- 
lated during instruction, the posttest, and retention test. 

Findings 

Table 1 includes the total states presented, total responses, total 
teaching time, and the student opinion survey mean score for the two experi- 
mental treatments. It can be seen from this table that there was very little 
difference in the two treatments based upon all of these criteria except 
total instructional time. Although there is some difference in the total 
number of states presented, there is virtually no difference in the total 
number of responses made in each experimental treatment. This means that 
in the gradient feedback treatment the student was responding more often 
to the same stimulus, but it was not necessary to present the stimulus as 
often as was required for the students in the full -response feedback program. 

An adjusted analysis of variance using the pretest scores as a covariant 
(Table 2) was performed on total instruction time. The pretest correlated 
-.847 with total instructional time. The analysis produced an F ratio of 
15.3 which was significant beyond the .01 level. 

Table 3 presents the mean score, mean response latency with standard 
deviations and adjusted mean scores of the independent adjusting variables 
and the dependent variables for treatment groups. For the analysis of the 
posttest data, the pretest scores and the typing scores were used as co- 
variants in an analysis of covariance. A Kuder- Richards on formula 20 relia- 
bility estimate of .725 was found for the posttest. The pretest correlated 
.468 with the posttest; the typing test correlated .403 with the posttest. 

The adjusted analysis of variance for the posttest scores resulted in an 
F ratio of less than unity which was not significant. 

An adjusted analysis of variance was also performed on the retention 
test scores (Kuder-RIchardson formula 20 reliability estimate of .893) using 
the posttest and the typing test as covariants. The posttest correlated .675 
with the retention test, and the typing test correlated .592 with the retention 
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test. This analysis resulted in an F ratio of 1.44, which does not reach 
significance at the .05 level of significance. 



Table 1 

Total Stimuli Presented, Total Responses, Total Instructional Time, 
and SOS Mean Score for the Two Experimental Treatments 



(Gradient-response Feedback) (Full -response Feedback) 
Mean S.D. Mean S.D. 

N=12 N=12 



Total No. of 


Stimuli Presented 


99.33 


28.67 


110.58 


45.46 


Total No. of Responses 
Highest 
Lowest 


180.42 

288 

70 


75.48 


178.00 

378 

70 


94.88 


Total Instructional 


Time in Minutes 


98.72 


38.64 


75.80 


32.18 


SOS Score 


72.88 


6.43 


72.75 


8.37 



Table 2 

Adjusted Analysis of Variance for Instructional Time 
with the Effects of Prior Knowledge Removed 



Source 


Adjusted 

d.f. 


Adjusted 
Sum of 
Squares 


Adjusted 

Mean 

Square 


F ratio 


Treatment 


1 


3682.75 


3682. 75 


15.3* 


Error 


21 


5051 .02 


240.52 




Total 


22 


8733.77 







♦Significant at the .01 level 
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Discussion 

Although no differences in learning (as measured by a performance test) 
were found between the experimental treatments, students can learn via 
computer-assisted instruction as evidenced by the mean on the pretest 
(19.21) and the mean on the posttest (43.91). The decrease in variability 
between pretest (s.d. = 12.62) and posttest (s.d. = 4.41) further indicates 
that students can achieve at least partial mastery of the material via CAI. 

Many problems occur in conducting experimental studies with human 
subjects in remote locations. Scores on retention tests administered three 
months after the initial learning (not reported in this paper) were con- 
siderably higher than scores on the two-week retention test. In discussing 
this matter with the proctors, it was discovered that the size of the small 
campuses apparently enabled the "grapevine" among subjects to be very fast 
and very effective. 

Other data collected during the course of the experiment seem to indi- 
cate that perhaps there were differences between the treatment groups. 

Table 4 presents the correlations of posttest scores and pretest scores, 
response latency during learning, total responses during learning, teaching 
time, and response latency during posttest for each experimental treatment. 
These are all variables which have potential for indicating learning in 
addition to posttest scores. 

The correlations between retention test scores and all other variables 
which might indicate other aspects of learning is presented in Table 5. 

There is very little difference in the correlations produced by the two 
treatments except in the response latency for posttest and response latency 
during learning. A scatter diagram of the response latency for posttest 
revealed very little information. However, a scatter diagram (Fig. 2) of 
the retention test scores and response latencies during instruction for 
subjects in the gradient feedback and full -response feedback treatment 
groups indicated that the full response feedback produced much stronger 
readiness to respond than did the gradient feedback treatment. The corre- 
lation between retention test scores and total response latency during 
instruction of -.505 for the full -response feedback treatment group and 
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Table 4 



Correlations Between Posttest Scores and Pretest Scores, 
Response Latency during Instruction, 

Total Responses during Instruction, Teaching Time, and 
Response Latency during Posttest for Each Experimental Treatment 







Correlations Between Posttest Scores and: 






Pretest 


Response 

Latency: 

Instruction 


Total 

Responses : 
Instructi on 


Instructional 

Time 


Response 

Latency: 

Posttest 


Gradient Feedback 
(n*12) 


.382 


-.492 


-.439 


-.516 


-.313 


Ful 1 -Response 
Feedback 
(n®12) 


.553 


-.480 


-.497 


-.512 


-.091 



-.229 for the gradient feedback group suggests that when ful 1 -response feed- 
back Is used, response latency can be a fairly good measure of student progress. 



Conclusions 

Although the treatment comparisons presented in this paper did not show 
statistically significant differences In learning, there was a very strong 
difference In Instructional time for the two treatments. The response latencies 
also show rather striking differences between the two treatment groups and hold 
some promise for directing future research. The anticipated advantage for 
gradient feedback over full -response feedback did not materialize In this study. 

We are well aware that additional Investigations will be required before discarding 
the theoretically sound Idea of providing learners with a gradient feedback. 

Because of the problems encountered with remote locations It was not con- 
sidered advisable to engage In further statistical analysis of the data although 
they do suggest a trend. Sllberman, Melaragno, Coulson and Estavan (1961) 
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Fig. 2. Scatter diagram of retention test scores and response latency 
during instruction for subjects in the gradient feedback and full -response 
feedback treatments. 
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conjectured that some measures such as response latency are more appropriate 
than error rate for making certain decisions within a program. Postman 
and Egan (1949) point out that reaction time remains a sensitive measure 
of the readiness of an organism to respond. Using data of this kind has 
always been a difficult process because of the technical problems involved 
in measuring response latency and feeding the information back into the 
operating system for decision-making purposes. However, with the develop- 
ment of computer-assisted instruction and the speed and flexibilities of 
such systems y this information is readily available and can easily be used 
for making such decisions. It is the intent of the authors to continue 
investigating these variables which seem to hold promise for improving 
learning with computer-assisted instruction. 
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